The Responsive Room

We introduce the Responsive Room, an interactive, ambient environment designed to augment spoken language with light and auditory cues by analyzing patterns in spoken speech. This paper focuses primarily on the use of the Responsive Room for storytelling. To this end we constructed a scripted, theatrical sequence that explores the relationship between story telling, color and sound. We also analyzed the possible automated generation of these sequences from analysis of the spoken speech. An offline analysis extracted important categories from the text such as affect, locale and topic. The system used to analyze speech for major story features is described, as is the theory for mapping these features to colors. The preliminary results of an exploratory experiment with children are discussed.